The increasing diversity of image display formats has resulted in considerable interest in converting images from one sampling structure to another. In particular the increased use of “high-definition” (HD) television formats (i.e. formats where the vertical spatial sampling frequency is of the order of 1,000 samples per picture height) has led to the development of production equipment which can operate on both HD and standard definition (SD) formats. In some equipment, processing is carried out at one particular high-resolution sampling structure, and inputs can be received, and outputs can be delivered, either with the structure used for processing, or at a different sampling structure.
In such equipment images may need to be converted to a higher or lower rate before or after processing. It is now common for two, complementary conversions to be needed: one from the input sampling structure to the processing structure; and another, from the processing structure back to the original input structure. It is highly desirable for these conversions to be transparent to the user, and, provided the processing structure has higher sampling frequencies in all dimensions, the cascaded up- and down-sampling can be made mathematically transparent.
Typically, input sample values from adjacent sample locations are combined as a weighted sum in a filter aperture so as to derive new sample values at locations other than the input sample locations. The filter may have several “phases” so that different weightings are used to obtain new values for sample sites having different spatial relationships to the input sample locations. Often the filter is one-dimensional so that the filter aperture comprises a set of horizontally contiguous samples or a set of vertically contiguous samples. However, two, and three-dimensional filters are also known, in which the filter aperture comprises a set of samples in a two-dimensional image region, or a set of two dimensional regions from images sampled at different times.
In multi-standard video processing systems it is desirable that complementary up- and down-conversion filters should be available with similar levels of complexity—i.e. aperture size and coefficient magnitudes. Symmetrical filters are easier to implement. Both types of filter should have high stop-band attenuation, so as to reduce aliasing; and, these features should be combined with the property of reversibility. Such filters are required for conversion between the commonly-used horizontal sample structures of 720, 960 and 1280 samples per line; and vertical sampling structure of 480, 486, 576, 720 and 1080 lines per picture. In these conversions the relative phasing of the input and output sampling grid may be required to be offset so as to avoid a shift in the position of the image centre when converting.
U.S. Pat. No. 6,760,379 (Werner)—which is hereby incorporated by reference—describes how, given a linear up-conversion filter, a complementary down-conversion filter can be derived, so that the cascading of the two complementary processes is mathematically transparent. It is helpful to review the example filters shown in this prior patent so as to clarify the improvements provided by the current invention.
FIG. 1 illustrates up-conversion by a factor of 3:4 followed by down-conversion by 4:3 according to a mutually reversible pair of filters. The set of low-resolution samples L0 to L4 is up-converted to the (more-numerous) set of high-resolution samples H0 to H5, which are then down-converted to the low-resolution samples L′0 to L′4. The up-conversion filter is a bilinear interpolator and the down-conversion filter is the filter ‘undo-bil’ described in the Werner patent.
In FIG. 1 the filter coefficients are shown by arrows. Following the notation of Werner, up-conversion filter coefficients are denoted by g(x) and down-conversion filter coefficients are denoted by h(x), where x is the phase offset between the input and output samples expressed in units of the pitch of an intermediate sampling grid which includes both sets of samples. In the Figure each down-conversion filter coefficient is only shown once, but up-conversion filter coefficients are shown for all samples which contribute to any samples contributing to the shown down-conversion coefficients.
The coefficients of the FIG. 1 filters are shown in FIG. 2 in the form of filter aperture functions. The filter aperture (20) corresponds to the bilinear up-conversion filter g(x), and the filter aperture (21) corresponds to the filter h(x), which is the filter ‘undo-bil’ described in the Werner patent.
As can be seen from FIGS. 1 and 2, the down-conversion filter ‘undo-bil’ is asymmetric. The prior patent describes how such an asymmetric filter can be converted to a symmetric filter by averaging the filter aperture function with a mirror-image-reversed version of the aperture function, without losing the property of reversibility. FIG. 3 shows the same up- and down-conversion process as FIG. 1, but using a symmetric down-conversion filter derived from the filter ‘undo-bil’. (Note that in FIG. 3 zero filter coefficients are not shown.) The corresponding down-conversion filter aperture is the function (41) shown in FIG. 4.
It can be verified from the coefficient values shown in FIG. 3 that the set of down-converted samples L′o to L′2 are identical to the original samples Lo to L2, i.e. the up-conversion has been reversed transparently:
                              L          0          ′                =                ⁢                              1            ×                          H              0                                =                      1            ×                          L              0                                                                        L          1          ′                =                ⁢                              [                                                            -                  1                                /                6                            ×                              H                0                                      ]                    +                      [                                          2                /                3                            ×                              H                1                                      ]                    +                      [                          1              ×                              H                2                                      ]                    -                      [                                          2                /                3                            ×                              H                3                                      ]                    +                      [                                          1                /                6                            ×                              H                4                                      ]                                                  =                ⁢                                                            -                1                            /              6                        ×                          L              0                                +                                    2              /              3                        ×                          (                                                                    1                    /                    4                                    ×                                      L                    0                                                  +                                                      3                    /                    4                                    ×                                      L                    1                                                              )                                                                      ⁢                              1            ×                          (                                                                    1                    /                    2                                    ×                                      L                    1                                                  +                                                      1                    /                    2                                    ×                                      L                    2                                                              )                                -                                                ⁢                                            2              /              3                        ×                          (                                                                    3                    /                    4                                    ×                                      L                    2                                                  +                                                      1                    /                    4                                    ×                                      L                    3                                                              )                                +                                    1              /              6                        ×                          (                              L                3                            )                                                              =                ⁢                              [                                                                                -                    1                                    /                  6                                ×                                  L                  0                                            +                                                1                  /                  6                                ×                                  L                  0                                                      ]                    +                                                ⁢                              [                                                            1                  /                  2                                ×                                  L                  1                                            +                                                1                  /                  2                                ×                                  L                  1                                                      ]                    +                                                ⁢                              [                                                            1                  /                  2                                ×                                  L                  2                                            -                                                1                  /                  2                                ×                                  L                  2                                                      ]                    +                                                ⁢                  [                                                                      -                  1                                /                6                            ×                              L                3                                      +                                          1                /                6                            ×                              L                3                                              ]                                        =                ⁢                  L                      -            1                              
Similarly, the contributions to L′2 are a mirror image of the contributions to L′1, and so L′2 is equal to L2.
And, all other down-converted samples are derived by contributions analogous to those used to derive L′0, L′1 and L′2.
The Werner patent describes a second pair of complementary up- and down conversion filters, referred to as ‘def’ and ‘undo-def’, where the up-conversion has an improved frequency response, as compared to bilinear up-conversion. The aperture functions of these prior art filters are shown in FIG. 5, in which the up-conversion aperture is designated (50), and the down-conversion aperture is designated (51).
Note that the Figure only shows the central portion of the down-conversion filter aperture (51) which contains the largest coefficients; the full aperture extends over x values in the range −8 to +20. The coefficients of the full aperture are given in Table 4, which also includes all the down-conversion apertures described in this specification. (Table 3 lists the coefficients of all the up-conversion apertures described in this specification.) The down-conversion filter aperture (51) can be made symmetrical by the same method as described above, and the resulting filter aperture is shown as the aperture (61) in FIG. 6. Again, only the central portion of the aperture is shown in the figure, the full aperture extends from −20 to +20. It can be seen that the complementary down-conversion filter apertures (51) and (61) are much wider than the corresponding up-conversion filter aperture (50) (also shown as (60)).
It is helpful to examine the frequency responses of these prior-art filters, and they are shown in FIGS. 7 to 10. The responses have been calculated by converting the relevant filter apertures, which, of course, are the respective filter impulse responses, from the time domain to the frequency domain in the well-known manner.
The frequency scales of FIGS. 7 to 10 are in units of 12 times the frequency of the notional sampling grid that contains both the lower and higher sampling frequencies; i.e. the lower sampling frequency (flow) corresponds to three units, and the higher sampling frequency (fhigh) corresponds to four units. The Figures show the magnitudes of the responses; however, for the symmetrical filters, for which the response is always real (i.e. its imaginary component is zero), phase-inverted responses are shown as having negative amplitude.
FIG. 7 shows the response (71) of the ‘undo-bil’ down-conversion filter of the Werner patent; and, the response (70) of a bilinear up-conversion filter.
FIG. 8 shows the response (81) of a symmetric filter derived from the ‘undo-bil’ down-conversion filter by the method described in the Werner patent; and, the response (80) of a bilinear up-conversion filter (identical to the response (70)).
FIG. 9 shows the response (91) of the ‘undo-def’ down-conversion filter of the Werner patent; and, the response (90) of the ‘def’ up-conversion filter also described in the Werner patent.
FIG. 10 shows the response (101) of a symmetric filter derived from the ‘undo-def’ down-conversion filter by the method described in the Werner patent; and, the response (100) of the ‘def’ up-conversion filter (identical to the response (90)).
It can be seen that the frequency responses of these prior art down-conversion filters are less than optimum. They all have poor stop-band attenuation (above 1.5 frequency units); in particular there is significant response at the lower sampling frequency flow (3 frequency units). Any signal energy at this frequency would be aliased to DC. This alias is likely to prove particularly troublesome when down-converting material which has not been previously up-converted. It can also be seen that none of the filter pass-bands are particularly flat.
The up- and down-conversion filters shown in FIGS. 1 to 6 are what the Werner patent describes as “Nyquist” filters for which a regular sub-set of the output samples are equal to input samples. The samples in this sub-set are formed by copying the values of respective, spatially-aligned filter input samples. For example, in FIG. 1 the up-conversion filter g(x) and the down-conversion filter h(x) both give outputs copied from their respective inputs when the input to output phase offset x is zero.